I. Introduction
In recent years, the development of the AI field has made great progress, and the related products incubated are in full swing. Especially ChatGPT, which has exploded recently, has made friends in other non-IT fields start to contact and use it. In fact, its explosion is not accidental, because ChatGPT is really powerful and can really solve problems in practical applications, so people will start to pay attention to it and use it.
In fact, there are also good large-scale model platforms in China. A few days ago, I also evaluated and shared the large-scale model platform developed by Baidu itself - Wenxin Qianfan , and the effect of using it exceeded expectations.
Just yesterday, I saw that the Baidu team has upgraded the Qianfan large model platform: full access to 33 large models such as the full series of Llama 2 , ChatGLM2-6B , RWKV-4-World , MPT-7B-Instruct , and Falcon -7B , becoming the platform with the most large models in China:
At the same time, it supports dual enhancements of performance and security, and the cost of model inference can be reduced by 50% . In addition, the Qianfan large-scale model platform has also launched a preset Prompt template library with 103 templates , covering more than ten scenes of dialogue, games, programming, and writing.
After seeing these upgrades, I couldn't wait to try them again. Let me share with you the process of using this new feature.
Article directory
1) Preset prompt template experience
2. Use and share
1 ) Preset prompt template experience
The word prompt may be unfamiliar to everyone, but in fact, every time we ask Wen Xin Yi Yan a question is a prompt. But asking questions is also a science. If you ask a clear and accurate question, the answer you get must be more accurate. For example: when you go to see a doctor because of stomach pain, you tell the doctor that I have an upset stomach, then the doctor must have a lot of interaction with you, and only after checking other organs within the stomach area one by one can he know that you have an upset stomach. If you tell the doctor directly that I have a stomachache, the doctor will treat it more targetedly and save a lot of intermediate processes.
Because this is what the prompt template does, Baidu's preset prompt template is equivalent to more accurate, clear, and standardized questions.
In many industry scenarios, optimizing prompts is a key project and relies heavily on experience. If you try to explore a standardized prompt template from scratch, it will take a lot of effort and time. The upgrade of Qianfan’s large model has launched 103 preset prompt templates. Baidu has opened up the experience of serving many internal and external customers, including the experience of prompt engineering experts, and formed a rich template library:
So what is the difference between using a prompt template and not using a prompt template? Do these prompt templates provided by Baidu meet our business scenario needs? Here I have verified the movies I am interested in and my more professional Python field:
1.1 Movie reviews
If we want to write a film review for the movie "Farewell My Concubine", we usually directly say to the model: "Write a film review for Farewell My Concubine":
In fact, the results of the answers I got are not bad. After all, the ability of Wenxin Yiyan is still very good, but after a closer look, the evaluation of the movie is not comprehensive and detailed enough, and I don’t want spoilers, and I hope that the movie’s Soundtrack, special effects, etc. do more evaluations, so you can use the prompt template to see what kind of effect it will have:
The movie review prompt template provided by Baidu is relatively clear and comprehensive, so the answers output by the language model are clearer and more comprehensive. There are no more spoilers. It also analyzes and introduces the movie "Farewell My Concubine" from various angles. This answer is more in line with my expectations.
1.2 Python interpreter
Here we start from the most basic hello world, to list serialization and bubble sorting to see the difference between the answers:
Hello world does not use templates
Hello world uses a template
Serialize without using templates
Serialization using templates
Bubble sort does not use templates
Bubble sort using templates
It can be seen that after using the prompt template of the Python interpreter, Wenxinyiyan will intuitively output the results and explain the returned results. If you don't use templates, you will give detailed explanations and examples of the code. At this time, it is more inclined to let us understand the meaning of the code rather than the presentation of the results.
After experiencing it, the preset prompt template provided by Baidu can indeed express the effect I want more accurately, and it is more in line with my expectations. If you want to experience the prompt template, you can visit the online test page, and then click "prompt template-preset template" to refer to the preset template for testing:
2) Model repository
On the template warehouse page, you can see the full series of LLaMA2, RWKV and other large models accessed by this upgrade. We can directly click on the deployment of the page to use them:
Friends may have the same question as I did when I first started: Why does Qianfan need to access so many third-party large models?
Now let’s talk about Wenxin Qianfan’s philosophy and goals: Qianfan platform is designed around the large-scale model application requirements of enterprises, aiming to provide enterprise users with a full-scenario, one-stop large-scale model development and service tool chain. The current open source large model ecosystem is developing rapidly, and a large number of high-quality third-party models have emerged, showing differentiated advantages in different task scenarios, parameter magnitudes, and computing power environments. The Qianfan team selects high-quality third-party models in the industry and seamlessly integrates with the platform so that enterprise users can quickly experience, test, and access services; it can be used in conjunction with large models such as Wenxinyiyan (ERNIE-Bot) to better meet different needs. Business requirements for subdivided scenarios.
Moreover, Wenxin Qianfan does not just connect to the third-party platforms casually. They are based on three dimensions (commercial availability, model effect, and model security), and these models will only be connected after passing layers of assessment. And in order to ensure the security of models used by enterprises and developers, Qianfan has made model security enhancements to all connected third-party models, not only ensuring the content security of Wenxin large models, but also ensuring the safe output of third-party large models .
In addition, in order to reduce the cost of use, Qianfan has made secondary performance enhancements for each connected large model. By optimizing model throughput and reducing model size, the speed of model inference is greatly improved. According to estimates, after tuning, the model volume can be reduced to 25%-50% , and the inference cost can be reduced by 50% . This means that compared with direct calls, enterprises can greatly save costs and improve results by using these models on the Qianfan platform.
Moreover, Qianfan has made in-depth adaptation to the large models connected, and provides a full set of tool chains for model retraining, supporting various forms of model tuning, including SFT (full parameter fine-tuning, Prompt Tuning , Lora ) and reinforcement learning (reward model learning, reinforcement learning training), etc. Help enterprises and developers to quickly retrain based on the basic large model, and build an enterprise-specific large model.
3. Summary
The effect of this experience is still very good. It surprised me again and made me realize that it is not "the moon will be round only when the moon is abroad". There are also good products in China, and even the support in some aspects is more complete. , the supported functions are also more comprehensive:
In addition to being comprehensive and safe, the Qianfan platform is also more efficient. In the test results of MLPerf Training v2.1 released in November 2022, the model training performance results submitted by Baidu using Flying Paddle plus Baidu Baige ranked first in the world under the same GPU configuration. First, the end-to-end training time and training throughput surpass the NGC PyTorch framework:
Moreover, Qianfan's service models are also diversified, supporting public cloud services and privatized deployments:
The public cloud provides three service models: inference, fine-tuning, and hosting. Applicable to enterprises and developers with different development capabilities and needs:
1. Reasoning: directly invoke the core reasoning capabilities of the general large model and output reasoning results.
2. Fine-tuning: Based on the basic capabilities of the general-purpose large model, customers can fine-tune their own large-scale model at a small cost by injecting a small amount of industry data according to their own needs.
3. Hosting: General large models or fine-tuned industry large models can be directly hosted on the cloud of Baidu Smart Cloud. Customers only need to use the large model, and Baidu Smart Cloud will ensure the high availability, high performance and high security of the large model, and enterprises do not have to worry about complex deployment and management issues.
Privatized deployment supports two delivery modes: pure software platform and integrated software and hardware:
- Pure software platform delivery: Serving large models running in an enterprise environment. Baidu Smart Cloud authorizes the packaged AI software system to customers, who build and deploy them in local data centers or private cloud environments. Baidu is responsible for software installation and debugging, training and maintenance support and other services. This model can maximize data privacy and control, but requires customers to have certain AI operation and maintenance capabilities and bear the corresponding server costs.
- Integrated delivery of software and hardware: Baidu Smart Cloud not only authorizes AI software systems to customers, but also provides pre-configured AI server clusters and storage systems for delivery. We will be responsible for the deployment, debugging and daily technical support and maintenance of the overall solution. We can provide end-to-end service and warranty, and also provide certain discounts on hardware prices.
Even with many advantages and the support of many firsts in the industry, we still have to admit that there is still a certain gap in the language model between many large-scale model products in China, including Wenxin Qianfan and ChatGPT, but I believe that in the future this With the efforts of all "siege lions" and the support of all parties, the gap will become smaller and smaller, and even overtake! I hope that the friends will give them more time and put forward some suggestions to make them better and better.
Friends can visit Wenxin Qianfan Large Model Platform Public Version Test Service to apply for a test to see if it can meet your needs.