Use FastGPT to build a high-quality AI knowledge base

Author: Yu Jinlong. FastGPT project author, Sealos project front-end leader, former Shopee front-end development engineer

FastGPT project address: https://github.com/labring/FastGPT/

introduction

Since the release of ChatGPT in December last year, it has driven a new round of interactive application revolution. Especially after the full opening of the GPT-3.5 interface, a large number of LLM applications have sprung up. However, due to the controllability, randomness and compliance of GPT, many application scenarios cannot be implemented.

origin

In March, I posted on Twitter that an old man used GPT to train his own blog record, and the cost was extremely low (compared to Fine-tuning). He provides a complete flowchart:

After seeing this tweet, I had an idea, and the application scenario became very clear. I started working directly. In less than a month, I added a vector search function to FastGPT on the basis of the original assistant management. So there is the earliest video: https://www.bilibili.com/video/BV1Wo4y1p7i1/

initial development

Three months have passed, and FastGPT still continues the early ideas to improve and expand. At present, its functions in vector search + LLM linear question answering have basically been completed. However, we never published a tutorial on how to build a knowledge base. Therefore, we intend to write an article to introduce "How to build a high-quality knowledge base on FastGPT" during the development of the V4 version.

FastGPT's knowledge base logic

Before officially starting to build the knowledge base, we need to understand FastGPT's knowledge base retrieval mechanism. First, we need to understand a few basic concepts:

basic concept

1. Vector : Convert human language (text, pictures, videos, etc.) into computer-recognizable language (array).
2. Vector Similarity : Computes the similarity between two vectors, indicating how similar two languages ​​are.
3. Properties of Large Models of Language : Contextual Understanding, Summarization, and Reasoning.

The combination of these three concepts constitutes the formula of "vector search + large model = knowledge base question answering". The following is the complete logic of the knowledge base question answering function in FastGPT V3:

What makes FastGPT different from most other knowledge base question answering products is that it uses QA question answer pairs for storage, not just chunk (text chunking) processing. This is done to reduce the length of the vectorized content, so that the vector can better express the meaning of the text, thereby improving the accuracy of the search.

In addition, FastGPT also provides two ways to adjust the data, search test and dialogue test, so that users can adjust their own data conveniently.

According to the above process and method, we take building a FastGPT FAQ robot as an example to show how to build a high-quality AI knowledge base.

FastGPT warehouse address: https://github.com/labring/FastGPT

Create a knowledge base application

First, we create a FastGPT FAQ knowledge base.

Basic knowledge acquisition

Let's first directly split some existing documents on the FastGPT GitHub warehouse for QA splitting, so as to obtain some basic knowledge of FastGPT. Let's take README as an example.

QA fixes

We obtained 11 sets of data from README, the overall quality is good, pictures and links are extracted. However, there is some truncation in the last knowledge point, and we need to fix it manually.

In addition, we noticed the third knowledge point in the first column, which introduced some resource links of FastGPT, but the QA split put the answer in A. However, users' questions usually don't directly ask "what are the links", they are more likely to ask: "deployment tutorial", "issue documentation", etc. Therefore, we need to deal with this knowledge point simply, as shown in the following figure:

Next, we can create an application and see how it works. First create an application and associate related knowledge bases in the knowledge base. In addition, you also need to tell GPT: "The scope of the knowledge base" in the prompt word on the configuration page.

Import Community FAQs

Next, we import the documents of the FastGPT FAQ. Since the previous arrangement was not in place, we can only manually enter the corresponding questions and answers.

The import result is as shown above. It can be seen that we all use the format of question and answer pairs, rather than rough direct import. The purpose is to simulate user problems and further improve the matching effect of vector search. Multiple questioning methods can be set for the same question, and the effect is better.

FastGPT also provides the OpenAPI function. You can process files in special formats locally and then upload them to FastGPT. For details, please refer to: FastGPT Api Docs

Knowledge base fine-tuning and parameter tuning

FastGPT provides two functions of search test and dialogue test, we can use these two functions to fine-tune the knowledge base and adjust parameters.

We recommend that you collect some user questions in advance for testing, and jump according to the expected effect. You can search, test and adjust first to judge whether the knowledge points are reasonable.

search test

Through the search test, we can enter questions and view the returned knowledge base data to test the query effect of the knowledge base. The following is the interface of the search test:

We can see that the system returned the question and answer data related to it.

You may encounter the following situation. Due to the keyword "knowledge base", the similarity of some irrelevant content is also searched. At this time, you need to add a "knowledge base" keyword to the fourth record, so that to increase its similarity.

Prompt word settings

The role of the prompt word is to guide the direction of the model's dialogue. When setting prompt words, follow two principles:

1. Tell GPT what aspect to answer.
2. Give the knowledge base a basic description, so that GPT can better judge whether the user's question belongs to the scope of the knowledge base.

Better limit the scope of model chat

First of all, you can limit the chat scope from the knowledge base level by adjusting the similarity and the maximum number of searches in the knowledge base. Usually we can set the similarity to 0.82, and set an empty search reply content. This means that if the user's question cannot be matched in the knowledge base, the preset content will be replied directly.

Since the OpenAI vector model is not aimed at Chinese, when there are some keywords in the knowledge base content in the question, the similarity will be higher, and at this time it cannot be limited from the knowledge base level. Adjustments need to be made via qualifiers, for example:

If my question is not about FastGPT, please reply directly: "I'm not sure". You only need to answer the content in the knowledge base, and the content that is not in it does not need to be answered.

The effect is as follows:

Of course, GPT-3.5 is still uncontrollable under certain circumstances.

Tuning the Knowledge Base Through Dialogue

Similar to the search test, you can directly modify the content of the knowledge base at any time by clicking "Reference" on the chat page.

epilogue

1. Vector search is a technique that can compare text similarities.
2. Large models have the ability to summarize and reason to answer questions from a given text.
3. The most effective ways of knowledge base construction are QA and manual construction.
4. The length of Q should not be too long.
5. The prompt words need to be adjusted to guide the model to answer the content of the knowledge base.
6. The scope of the model's responses can be controlled by adjusting the search similarity, maximum number of searches, and qualifiers.