Author: Han Xie
Author: Han Xie
AIGC continues to be popular all over the world, and more and more scenarios are beginning to connect and reflect extraordinary value. One of the widely used scenarios is intelligent knowledge question answering, which has changed the way people learn, from reading to question answering, making knowledge acquisition more accurate and effective. Open source software has a wide range of knowledge-seeking groups. The combination of the AIGC+ community is a novel attempt in the future evolution of the learning community. Today we will talk to you about the experience of building this type of scenario and the problems we encountered based on the practice of the RocketMQ learning community. some challenges.
A New Paradigm for Learning Communities
The appeal of the learning community itself is to quickly disseminate knowledge and increase influence, while community developers hope to obtain professional knowledge faster, more timely, and more accurately.
In the past, there were two ways to obtain knowledge from the community:
- Read the community's documentation and source code provided by the community for independent learning;
- Communicate with the technical experts in the community to get answers.
There are certain problems in both methods. The cost of the first self-study is relatively high. In order to better understand the related basic concepts, you need to follow the quick start document step by step, and then continue to learn other knowledge points. If you want to get more The cost of knowledge concerned is high, and due to differences in individual understanding, the effect of community dissemination of knowledge may not be satisfactory; although the second type can accurately obtain the desired knowledge, the mastery of individual knowledge is better than the first type , but the answers from community experts are often not real-time, so neither the learning of developers nor the dissemination of community knowledge is efficient.
Now there is a third way: professional knowledge question and answer based on large language model. Community developers can ask professional knowledge questions anytime and anywhere, and enjoy 24-hour expert service. For the learning community, the efficiency of disseminating their own knowledge has also become higher up.
Basic principles of intelligent question answering technology based on proprietary corpus
As shown in the figure, the intelligent question answering system of the proprietary corpus is divided into two parts:
-
For the entry of the corpus, the administrator uploads professional community materials, and the intelligent question answering system slices the documents, and then converts them into vector data through the embedding algorithm, and stores them for retrieval;
-
Question answering, the question entered by the user is converted into a vector and compared with the content of the vector database through an approximate search algorithm. After obtaining an approximate answer, it is input into the large language model combined with the prompt word template and user questions, and the large language model is summarized. returned to the user. Therefore, the complete function of the system includes both the direct answer output (the result may not be accurate according to the ability of the large model, mainly "phantom answer"), and the original text list retrieved based on the vector approximation. The latter serves as supporting evidence to help learners judge whether the answer is correct or not.
production challenge
Compared with the demonstration of technical solutions, there are many challenges in really wanting to release intelligent question answering capabilities to production. Let's discuss with you below.
Safety
Security is always the most important concern of AIGC-like services, and there is no one. It mainly includes the following points:
- Data Security
- Content Safe, contains output and output
- system security
Data security mainly refers to the use of large-scale model services, especially overseas services will lead to cross-border data, which is not allowed;
Content security mainly means that user input and result output cannot contain pornographic or terrorist content;
System security is that the attacker can induce the disclosure of some confidential information through prompt words. All of the above security issues need to be seriously concerned.
Security Issues Solutions
The solution to data security is to use fully compliant large model services or fully managed open source models to achieve 100% privatization. The RocketMQ learning community is based on the open source chatlm6b question answering system solution and uses a variety of Alibaba Cloud product portfolios to achieve model services and AI application with integrated business logic.
chatlm6b question answering system solution:
Content security, with security algorithm filtering for input content, Alibaba Cloud’s green network service can well support this, avoiding the problem of non-compliance with user input, and the constraint of output results is completed through the prompt word project-only answers are allowed problems in the field.
System security, to deal with the risk of prompt word injection (PI), you can consider adopting heuristic methods to filter malicious input before it arrives or use a dedicated LLM to analyze incoming prompts and identify potential attacks.
Related solutions can refer to: https://rebuff.ai/
High service availability
After the community's intelligent question answering system goes online, it will inevitably face more access requests. Model reasoning itself cannot be parallelized, which means that requests entering model reasoning need to be processed in a queue. How to quickly pull up more model services and deal with concurrent requests is a real problem faced by online production; in addition, how to deal with single-point service failures and how to process data Disaster recovery and backup are issues that must be considered in production services.
Balance of cost/efficiency/experience
How to deliver business services faster and how to achieve a balance between experience and cost are also issues in actual production.
If you choose the self-hosting model, there are two options for using GPUs. The first is to buy a card to build the service yourself, and the second is to rent a server from a cloud vendor. The first type requires a strong technical background and needs to handle the high availability of services. The second type requires careful calculation of GPU consumption. The cost of GPU services provided by any cloud vendor is not low. In addition, as the business delivery time approaches, dealing with non-business technical issues will also increase the risk of business delivery, and the reduction of business R&D investment will also affect the service experience.
In one sentence, it is not easy to achieve a balance among cost, efficiency, and experience.
Exploring the RocketMQ Learning Community
The construction of the RocketMQ learning community adopts the Alibaba Cloud Serverless architecture to achieve 100% privatization and to solve security issues as much as possible. The balance of cost/efficiency/experience and high availability of services are achieved through Serverless. In addition, a more secure vpn intranet service is used at the network level to prevent the leakage of key system information to a greater extent. At the same time, it also retains the ability to switch business model services, so that more accurate answers to questions can be obtained during subsequent switching.
RocketMQ Learning Community Serverless Architecture Solution
Evaluation results of the safety water level by the professional safety team
Open source model address
https://github.com/imClumsyPanda/langchain-ChatGLM
Summarize
In addition to the above production challenges, we also encountered a lot of detailed problems in engineering research and development, such as serverless architecture model service deployment, end-to-end performance tuning, and we will disassemble them one by one with you in the future to make more detailed articles share.
We believe that the emergence of large models continues to change the world. With the maturity of intelligent question answering technology based on professional corpora, a wider range of industries such as education, medical care, and law will also benefit. The open source community will also usher in a new direction. Today, we will share our practice with you. We also hope that more students can participate in it, create together, and amplify the ability of the open source community to spread knowledge. More developers from HP .
RocketMQ learning community experience address
The RocketMQ learning community is the first knowledge service community based on AIGC in China. You are welcome to click to experience it (it is recommended to experience the full function on the PC): https://rocketmq-learning.com/ **
PS: The RocketMQ community uses RocketMQ 5.0 data as the main training content. During continuous optimization iterations, the answers are all generated by artificial intelligence models. The accuracy and completeness of the answers cannot be guaranteed, and they do not represent the attitude or opinion of the RocketMQ learning community.
Click here to experience the RocketMQ learning community now