This article is compiled from the keynote speech "Baidu Intelligent Cloud Qianfan Large Model Platform 2.0 Product Technology Analysis" by Xin Zhou, general manager of Baidu Intelligent Cloud AI & Big Data Platform, at the Baidu Cloud Intelligence Conference - Intelligent Computing & Large Model Technology Sub-forum on September 5, 2023. 》.
This is a forum about technical topics. I will first ask you three small questions from developers.
The first question: Who is the inventor of the steam engine?
Was it Watt, the famous developer in the 18th century? In fact, it was Newcomen who was 60 years earlier than Watt. Watt made a lot of improvements on the basis of the Newcomen steam engine, greatly improving its efficiency and starting the first industrial revolution.
Continuing with the question, do you know who invented the generator?
Faraday invented the world's first direct current generator in 1831, and more than 50 years later, in 1887, Tesla invented the alternating current generator. AC generators make power transmission more efficient and therefore can be transmitted farther. With the popularity of AC, the process of the second industrial revolution has been greatly accelerated.
Last question, what was the name of the first computer in the world?
ENAIC was born in 1946 at the University of Pennsylvania. More than 10 years later, in 1959, with the invention of the integrated circuit by Noyce of Fairchild Corporation, computers began to spread on a large scale and became an important cornerstone of the third industrial revolution.
Everyone must have guessed why there are these three questions. When a technology is invented and it is actually applied on a large scale, it will definitely go through a process of efficiency improvement, cost reduction, and large-scale popularization.
In the era of large models, Baidu Intelligent Cloud Qianfan Large Model Platform is committed to promoting this process, greatly improving the efficiency of large model development and application, reducing costs, and promoting industrial application and innovation.
On March 27 this year, we released version 1.0 of the Qianfan large model platform. Many corporate developers from various industries have approached us, hoping that both parties can discuss the application and implementation of large models.
Up to now, our Qianfan large model platform has more than 10,000 enterprises and users trying and exploring on it. We have tested more than 400 scenarios and developed solutions for multiple industries such as government affairs, finance, industry, transportation, etc. At the same time, we have also provided easier-to-use product tools in each link based on the problems that arise in the actual training and use of large models by enterprises. and more stable technical performance.
So today I am very happy to introduce to you the latest upgrade of Qianfan Large Model Platform 2.0.
At the MaaS layer, including the Wenxin large model, we have accessed a total of 42 large models with unique characteristics to meet the diverse needs for large models in all aspects of industrial application scenarios. We have improved and enhanced the large model full life cycle tool chain - by following step by step on this platform, you can quickly build an application you want or reconstruct your current product.
In addition to using large models, leading companies in many industries will use our platform to train large models. At the PaaS layer, we combine the capabilities of the AI development platform to achieve training acceleration, scenario modeling, application integration and other functions, providing best practices for enterprises training large models.
At the IaaS layer, Baidu Baige provides high-performance and stable AI infrastructure.
Below I will explain our ability upgrades one by one to everyone.
At present, the Qianfan large model platform provides 42 large models with different characteristics. In addition to the Wenxin large model, there are also ChatGLM of Zhipu Huazhang, which is very excellent in the market, and RWKV, which can support very large windows. There are also excellent foreign models such as BLOOMZ, Llama 2, etc.
In addition, we provide 41 data sets, including general data, special data, and instruction data, covering education, finance, law, etc. By using the data sets pre-installed on the Qianfan large model platform, developers can significantly reduce the data cost during training, especially during the cold start phase, and quickly build their own industry models.
In addition, we have made many enhancements to various third-party models based on the actual needs of the enterprise, one of which is Chinese enhancement. For excellent open source models like BLOOMZ and LIama 2, after they come to China, everyone will find that they are a bit acclimatized and cannot understand Chinese. Baidu uses its many years of Chinese data accumulation and Chinese knowledge to enhance these models in Chinese.
We can see that whether it is 7B or 13B parameters, the effect of Llama 2 after Chinese enhancement on the Chinese data level and various evaluation criteria are more than 10% improved compared to the original version.
At the same time, the application of the model is very resource intensive. We have compressed the size of the model and enhanced the inference speed of the model. The Qianfan Large Model Platform can reduce the size of open source models by more than 60% on average and increase the inference speed by up to 5 times, which is a great benefit for the practical application of our models.
In addition, we also provide enhanced capabilities such as command enhancement, performance enhancement, 32K context expansion, and security enhancement to meet the daily and long-tail scenario needs of enterprises.
As mentioned just now, Baidu Baige provides a high-performance and stable heterogeneous computing platform for upper-layer model platforms and model applications.
The Baige platform has made many optimizations in three aspects: training stability, fault awareness and fault tolerance.
In terms of training stability, the effective training time of Baige's Wanka task accounted for 95%.
At the fault awareness level, we build awareness capabilities for common fault scenarios such as task exit, task suspension, and slow operation. Especially the latter two faults are relatively concealed. The Baige platform has developed an indicator system based on a large number of best practices within Baidu. It can detect problems in seconds, locate faults in minutes, and complete fault recovery within 30 minutes.
Fault tolerance is the last hurdle to build stability. The Baige platform provides automatic fault tolerance, writes 100 GB checkpoints in seconds, and increases the effective training time by 10%.
In addition, for high-frequency scenarios of large-model applications, the Qianfan platform provides sample rooms for application paradigms, which can lower the threshold for enterprise application implementation. Taking the scenario of domain knowledge enhancement as an example, we introduce the entire process.
First, look at the far left side of the picture. The query entered by the user is passed to the large model through the API gateway. The large model disassembles the query into different subtasks. In this link, we can set or learn the relevant intention disassembly capabilities in advance by statically arranging the Chain or dynamically arranging the Agent. In addition, for the disassembled subtasks, automatic prompt optimization can also be further performed to more accurately pass them to the downstream.
In this scenario, the user's query may be broken down into domain knowledge retrieval, search enhancement, and a series of tool calls and queries. In this step, different subtasks are executed respectively, relevant tool calls and queries are completed, and the return results of the subtasks are obtained.
The bottom of the figure shows the system where knowledge enhancement in the domain will be called by subtasks. For example: we vectorize domain-related knowledge bases and documents in advance and store them in the database BES of Baidu Smart Cloud. As a retrieval analysis engine, BES (Baidu Elasticsearch) has comprehensively upgraded its vector capabilities in the era of large models, providing knowledge and memory for large models. While improving business performance, it can also effectively protect the security of corporate private domain data.
In the previous step, each sub-task obtained the returned results through vector database query, tool call, etc., and then input these structures into the large model for content processing and integration. Finally, the results after integrating the large model are filtered by our content security module and returned.
On the right side of the picture are the various infrastructures we rely on to build the entire system, such as key management, log management, etc.
The entire application sample room has two very distinctive features: The first feature is that the content is very comprehensive. The vector index includes search enhancement and SQL enhancement. We provide all of these tools that require domain knowledge retrieval. The second point is to support the rapid construction of applications, API gateway, LLM high-speed cache, key management and other functions necessary for enterprise-level applications. Users can directly use these functions through this sample room to quickly build based on their own applications and data. Build your own enterprise-level large-model applications.
More than a dozen model rooms like this have been provided on the Qianfan model platform to help enterprises and users quickly build their own generative AI applications.
Seeing is better than hearing a hundred times. Having said so much just now, I still hope to show you how to use the Qianfan platform. Let's watch a video to see how an engineer reconstructed an enterprise's data analysis product in 7 hours.
Our team has a summer intern classmate. The video records how he used the Qianfan large model platform to quickly build a generative AI application.
Friends who are very familiar with the development and application of large models will definitely be able to see that what he does is actually a function of interactive exploration of data. In the video, he did two things: the first thing was to fine-tune the instructions for querying from natural language processing to SQL statements; the second thing was to use knowledge retrieval in the domain to do questions and answers on professional knowledge in this field, and finally in the DEMO These two functions are debugged inside.
In the main forum this morning, Dr. Shen Dou launched our AI native application Family, in which service marketing, office efficiency improvement, and production optimization are all pre-installed with the Qianfan component version. Behind this capability is our overall full-site API plan, which allows enterprises and manufacturers of large model applications to more flexibly integrate Qianfan large models and tool chains into their own applications through the full-site API, and build their own AI native applications.
Taking the BI and data visualization product Sugar BI as an example, by calling the interface of Qianfanshangwenxin large model, based on traditional BI, it supports conversational data exploration, quickly obtains data charts and data conclusions, and can be applied to reports and large screens making. Sugar BI has received POC requests from dozens of customers within one month of its launch, which shows the market's enthusiasm for AI native applications.
AI computing power is developing very fast, and updates are commonplace. At the same time, there are already a very rich selection of AI computing power on the market.
However, so much AI computing power is a nightmare for the development and application of large models. Adapting various hardware is a technically difficult task that requires engineers who are familiar with hardware, frameworks, algorithms, and have rich experience in computing, storage, network and other fields.
The Qianfan large model platform has been adapted to the combination of mainstream computing power and models, and can manage and schedule different computing power, greatly improving the development efficiency and resource utilization of enterprises.
Currently, large models are in the early stages of industrial implementation, and high-quality data is a key element for realizing the industrialization of large models.
Massive data training, instruction fine-tuning, and reinforcement learning based on human feedback can continuously align large models with human values and ways of thinking, making large models more usable.
Baidu Intelligent Cloud independently developed the industry's leading large model data annotation platform, providing data services and operations, and can get through the last mile of large model implementation. The platform supports a closed-loop data production from data collection and cleaning, instruction fine-tuning annotation and reinforcement learning annotation to model evaluation.
In order to ensure the quality of data annotation, we have also built a full-process data service talent echelon and trained hundreds of full-time large model data annotators at the Baidu Smart Cloud Haikou data annotation base, with a 100% undergraduate rate.
Data security is always a top priority. Baidu Smart Cloud can provide high-security end-to-end data services. The annotation platform supports private deployment and provides customers with diversified data security solutions by linking with base resources.
At present, we have practiced with many companies and accumulated more than 400 scenarios. Including pan-technology, finance, energy, government affairs, etc., we will conduct in-depth cooperation with more industries in the future, allowing large models to empower thousands of industries.
In today's content, we introduce the latest upgrade of Baidu Intelligent Cloud Qianfan large model platform, including: models and data sets, tool chains and application paradigms, AI infrastructure Baidu Baige, full-site API, multi-core adaptation and other functions or characteristics. It also launched a key element to meet the industrial application of large models - a large model data annotation platform.
We hope that in the future, these products can help corporate partners further reduce the development and application costs of large models, jointly promote the industrial implementation of large models, jointly promote industrial innovation, and accelerate the intelligence of the industry.
Thousands of sails compete to create brilliance together!
- - - - - - - - - - END - - - - - - - - - -
Recommended reading
Introduction to the practical application of code understanding technology
The road to optimization of small program compiler performance
Optimization practice of Baidu APP iOS package size 50M (6) Useless method cleaning
Real-time interception and problem distribution strategy based on abnormal online scenarios
The author of the open source framework NanUI switched to selling steel, and the project was suspended. The first free list in the Apple App Store is the pornographic software TypeScript. It has just become popular, why do the big guys start to abandon it? TIOBE October list: Java has the biggest decline, C# is approaching Java Rust 1.73.0 Released A man was encouraged by his AI girlfriend to assassinate the Queen of England and was sentenced to nine years in prison Qt 6.6 officially released Reuters: RISC-V technology becomes the key to the Sino-US technology war New battlefield RISC-V: Not controlled by any single company or country, Lenovo plans to launch Android PC