Baidu Shen Dou: Upgrading “cloud-intelligence integration” to create a “super factory” for large model services

⭐Foreword

On October 17, Baidu World 2023 was held in Beijing Shougang Park. Baidu Intelligent Cloud announced during the conference that it has comprehensively upgraded its "cloud-intelligence-in-one" strategy and provided full-stack service solutions for five types of customer needs for implementing large-scale models; for AI native application development, it has released the "Qianfan AI native application development workbench" , accelerating the implementation of enterprise AI native applications; releasing the first domestic AI native application store and the first domestic large-model full-link ecological support system to empower partners’ business growth and jointly build and share a prosperous large-model industrial ecosystem.
At this conference, Shen Dou, executive vice president of Baidu Group and president of Baidu Intelligent Cloud Business Group, said that the deep integration of artificial intelligence and cloud computing is the key for enterprises to quickly implement native AI applications. This is also what Baidu Intelligent Cloud has always advocated and practiced. The concept of “cloud and intelligence integrated”.

⭐Five types of requirements for customer-oriented large-scale model implementation

Currently, all Baidu Group's applications and services run on Baidu Smart Cloud based on the "cloud-intelligence-in-one" technology architecture. In addition, in response to the five types of customer needs for implementing large models, Baidu Intelligent Cloud's "Large Model Super Factory" based on the Qianfan Large Model Platform provides the following five best service solutions.

1. For customers who only need computing power , the Qianfan platform can provide extremely efficient and cost-effective heterogeneous computing services. In the large model training process that customers are most concerned about, through distributed parallel training strategies and microsecond-level interconnection capabilities, the Qianfan platform can efficiently achieve large-scale expansion of computing power. The acceleration ratio of Wanka-scale cluster training reaches 95%; through prior prevention , discover, locate and solve problems in a timely manner to minimize the invalid operation of the cluster due to faults and other reasons, and increase the proportion of effective training time. The effective training time of the Wanka cluster exceeds 96%, fully releasing the effective computing power of the cluster, and significantly Reduce customer computing power and time costs. In addition, the Qianfan platform is also compatible with domestic and foreign mainstream AI chips such as Kunlun Core, Ascend, Haiguang DCU, NVIDIA, and Intel, allowing customers to complete computing power adaptation with minimal switching costs.
   Face Wall Intelligence teamed up with Zhihu to train the "Zhihaitu AI" large model and the multi-modal large model Luca based on the AI ​​computing power cluster provided by Baidu Intelligent Cloud. The effective training time on the kilocalorie cluster accounted for as high as 99%. While ensuring the continuity of model training, efficient convergence of model training effects can be achieved. In addition, Zhihu, TAL, Horizon, NetEase Youdao and other companies are also using the AI ​​computing services provided by Baidu Smart Cloud to achieve large-scale cluster training and management in a more stable, efficient and economical way.
2. At the model level, for customers who wish to directly call existing large models , the Qianfan platform manages 42 domestic and foreign mainstream large models. Enterprise customers can quickly call the APIs of various large models, including Wenxin large models, and obtain Large model capabilities. For third-party large models, the Qianfan platform has also made targeted optimizations such as Chinese enhancement, performance enhancement, and context enhancement. For example, large foreign models such as Llama2, which were originally better at English dialogue, have performed equally well in Chinese after Chinese enhancement. At present, the Qianfan platform has served more than 17,000 customers, and the number of large model calls continues to increase at a rate of 20% week-on-week.
3. For customers who want to conduct secondary development based on existing large models , the Qianfan platform provides a full life cycle tool chain and the industry's largest 41 high-quality industry data sets for retraining, fine-tuning, evaluation, and deployment of large models to help Customers can quickly optimize model effects for their own business scenarios. At present, many leading customers in the industry, including the Postal Savings Bank of China, Du Xiaoman, Kingsoft Office, and Hebei Hi-Speed ​​Group, have developed exclusive large models that meet business needs through the tool chain services provided by the Qianfan platform.
4. At the application level, some enterprises need to develop AI native applications based on large model services . The series of capability components and frameworks provided on the Qianfan platform can help enterprises quickly complete application development and flexibly respond to user and market needs.
5. Another group of customers hope to purchase mature AI native application products directly and conveniently to empower business development.

⭐ "Qianfan AI Native Application Development Workbench" accelerates the implementation of enterprise AI native applications

In order to meet the needs of enterprises for agile and efficient AI native application development and operation and maintenance, and to lower the threshold for AI native application development, Baidu Smart Cloud has launched the "Qianfan AI Native Application Development Workbench" , which will develop common patterns for large-model applications, Tools and processes are integrated into a workbench to help developers focus on their own business without involving unnecessary energy in the development process. Specifically, Qianfan AI native application development workbench mainly consists of two layers of services: application components and application framework.
The application component service consists of two major categories of components: AI and basic cloud components. It is a componentized encapsulation of the underlying service capabilities, allowing each component to complete a specific function. "AI components" include not only large language model components such as question and answer, Chain of Thought (CoT, Chain of Thought), but also multi-modal components such as Vincent diagrams and speech recognition, while "basic cloud components" include vector databases, objects Storage and other traditional cloud service capabilities.
The capabilities of many components here have been gradually accumulated by Baidu over the past 10 years, making it easier for us to develop large-model applications. Of course, if we use these components directly, there are still some thresholds and we need to have a certain foundation. Therefore, Baidu connects and combines these components so that they can complete the task of a specific scenario relatively completely. This is the application framework .

⭐Commonly used AI native application frameworks

Currently, the Retrieval Enhanced Generation (RAG) and Agent (Agent) provided on the Qianfan platform are commonly used AI native application frameworks. Under each framework, we can use the rich sample rooms provided by Baidu Smart Cloud to carry out agile and efficient AI native application development. Let’s first look at the RAG application framework.

⭐Retrieval Augmentation Generation (RAG)

The Retrieval Augmented Generation (RAG) framework can more efficiently utilize knowledge in an enterprise's proprietary fields and provide accurate answers to relevant questions with the help of large models. It is a must for native AI applications in the field of professional knowledge question and answer. Prepare core competencies.
We all know that in actual business, users often ask some very professional or highly targeted questions, and the answers are not on the Internet, but in the company's own document data. In this case, the general large language model cannot give an exact answer. This is a typical application scenario of RAG, which allows large models to learn and understand these specific professional knowledge and return accurate answers to users.
At the conference site, a practical demonstration was also conducted on how to quickly develop a knowledge question and answer application for Sany Heavy Industry based on the RAG framework: Just select the prefabricated RAG framework in the Qianfan AI native application workbench and perform corresponding parameter configuration and other work. Quickly realize the development and launch of the intelligent customer service application on the official website of Sany Heavy Industry.

The video first introduces the application scenario of the demonstration, then implements the development of intelligent customer service on the official website of Sany Heavy Industry in just a few minutes, and finally demonstrates the effect on the official website.

Let’s first take a look at the application scenario of the demonstration. This is the official website of Sany Heavy Industry. You can see that Sany Heavy Industry produces a variety of construction machinery and equipment including excavators. When a user comes to the homepage, he wants to consult about these equipment. When you have questions about performance parameters, operation and maintenance, etc., you have to find a staff member who is very proficient in these equipments in order to serve the user well. However, Sany Heavy Industry's documents cover a lot of content. For example, this is the operation and maintenance manual for an excavator called SY305H. One document has 48 pages and more than 20,000 words. It includes a lot of parameters and details. If you let an It is still very difficult for employees to flexibly grasp these contents and answer these questions from users. Baidu's RAG-based application framework can create a large model in a few minutes, allowing it to be quickly mastered and answer users' questions.
How to implement this function is actually very simple and only requires three steps. First, we enter the console of the Qianfan platform, build a knowledge base, and enter a name. After having the knowledge base, we load the document of the SY305H operation and maintenance manual.

Then enter the development application and click Plug-in Orchestration. In the configuration, we need to associate the knowledge base we just created. Now click OK to go online after the association is completed.

Now this knowledge inventory can be served online. The code is automatically generated here. We can copy it and paste it into the script on the homepage of Sany Heavy Industry’s official website. Go back to the homepage and refresh it. A small icon of Sany products will appear in the lower right corner. Assistant, you can now ask it specific questions about this excavator.

We can see that the assistant’s answer is very concise, what should be used and what should not be used, otherwise what kind of problems will it cause. We found out the content of this aspect in the document and compared it with the assistant’s answers. We found that the assistant is not only concise, but also very logical. It is precisely because this large model has such powerful language understanding and generation capabilities, so at this time we Just ask it a few other questions and you can still sort out the content.
A 20,000-word article and 48 pages of content can be turned into an online assistant in just a few minutes. This is the charm of large models.

⭐Agent

Agent (Agent), as a popular application framework in the current industry, can automatically disassemble tasks given by humans, automatically plan and call various components to collaboratively complete tasks, and at the same time receive feedback based on the results of task completion to improve its own capabilities. At present, the Agent framework has been widely used in industry, transportation and other fields.
Based on the Agent framework provided by Qianfan AI native application development workbench, Zhongtian Steel has created an intelligent "enterprise scheduling center" to realize the automatic perception, decomposition and execution of task instructions. For example, when it is discovered that steel production is not up to standard, the user only needs to ask once, and the large model can automatically call various resources and APIs managed by the platform to complete BI data retrieval, third-party root cause analysis, etc., find the reasons for not meeting the standard, and Make timely adjustments to production schedules.

⭐Summary

The era of large models is here, and it is turbulent. Some people are vying to ride the tide, while others are busy chasing the waves. Baidu is committed to building a solid ship, using more efficient intellectual infrastructure, a better-used one-stop large model platform, Richer industry solutions and AI native applications blooming.
Through the demonstrated case of Sany Heavy Industry and the understanding of the access process of Qianfan platform, in fact, it only takes a few steps to quickly develop an application. For enterprises, this means reducing costs and increasing efficiency; for technicians, Qianfan is great The rich tool set of the model platform provides the possibility of rapid development, and we can also focus our time and energy on more creative things.